Imaging method and apparatus

ABSTRACT

Disclosed is an imaging method for imaging terrain using a sensor on an unmanned aircraft. The method comprises: acquiring a range of motion of the sensor; acquiring positional information of the terrain; acquiring parameter values relating to aircraft maneuverability; using the acquired information, determining a procedure; performing, by the aircraft, the procedure and simultaneously capturing, by the sensor, a set of images of only parts of the terrain. The procedure comprises the aircraft moving with respect to the area of terrain and the sensor moving with respect to the aircraft such that each point in the area of terrain is coincident with a footprint of the sensor on the ground for at least some time. Also, every point in the area of terrain is present within at least one of the captured images.

RELATED APPLICATIONS

This application is a national phase application filed under 35 USC §371 of PCT Application No. PCT/EP2014/076538 with an Internationalfiling date of Dec. 4, 2014 which claims priority of GB PatentApplication No. 1321549.6 filed Dec. 6, 2013 and EP Patent ApplicationNo. 13275299.9 filed Dec. 6, 2013. Each of these applications is hereinincorporated by reference in their entirety for all purposes.

FIELD OF THE INVENTION

The present invention relates to imaging large areas of terrain usingsensors mounted on unmanned aircraft.

BACKGROUND

Unmanned Air Vehicles (UAVs) are commonly used to perform a variety oftasks. Such tasks include performing wide area searches of areas,surveillance operations, delivery of payloads, etc.

Conventionally, procedures to be performed by a UAV in order to completea task are determined by a human operator and typically involve thedirect control, by the human operator, of the UAV and on-board sensors.Such procedures may include, for example, the remote flying of the UAVby the operator to follow a route, and/or the moving of on-board sensorsetc.

Furthermore, typically data gathered by a UAV (e.g. using on-boardsensor systems) is transmitted to an entity remote from the UAV foranalysis.

However, the manual control of a UAV and the transmission of datagathered by that UAV tend to require relatively high band-widthcommunication.

SUMMARY OF THE INVENTION

In a first aspect, the present invention provides an imaging method forimaging an area of terrain using a movably sensor mounted on an unmannedaircraft, the method comprising: acquiring, by one or more processors, aspecification of possible positions and orientations relative to theaircraft to which the sensor may be moved; acquiring, by the one or moreprocessors, positional information of the area of terrain; acquiring, bythe one or more processors, a specification of the maneuverability ofthe aircraft; using the acquired specification of the possible positionsand orientations of the sensor relative to the aircraft, the acquiredpositional information of the area of terrain, and the acquiredspecification of the maneuverability of the aircraft, determining, bythe one or more processors, a first procedure and a second procedure;performing, by the aircraft, the first procedure, the first procedurecomprising the aircraft moving with respect to the area of terrain alonga first route and the sensor moving with respect to the aircraft suchthat, for each point in the area of terrain, that point is coincidentwith a footprint of the sensor on the ground for at least some timeduring the first procedure; whilst the aircraft performs the firstprocedure, capturing, by the sensor, a first set of images, each imagein the first set being of only part of the area of terrain, the firstset of images being such that, for every point in the area of terrain,that point is present within at least one of the images in the firstset; thereafter, performing, by the aircraft, the second procedure, thesecond procedure comprising the aircraft moving with respect to the areaof terrain along a second route and the sensor moving with respect tothe aircraft such that, for each point in the area of terrain, thatpoint is coincident with a footprint of the sensor on the ground for atleast some time during the second procedure; and, whilst the aircraftperforms the second procedure, capturing, by the sensor, a second set ofimages, each image in the first set being of only part of the area ofterrain, the second set of images being such that, for every point inthe area of terrain, that point is present within at least one of theimages in the second set. The first route is different to the secondroute. Thus, points on the ground tend to be imaged from differentdirections which tends to reduce the uncertainty of geolocations of thecaptured image points.

The method may further comprise registering the first set of images withthe second set of images.

The first set of images may be assembled into a first composite image ofthe area of terrain. The second set of images may be assembled into asecond composite image of the area of terrain.

Registering the first set of images with the second set of images mayinclude registering the first composite images with the second set ofimages.

For a region within the area of terrain, a direction in which theaircraft flies while that region is imaged during the first proceduremay be different (e.g. may be substantially perpendicular) to adirection in which the aircraft flies while that region is imaged duringthe second procedure.

The method may further comprise processing the first set of images todetect, within at least one of the first images, a first target, andacquiring, by the one or more processors, a position on the ground ofthe detected first target. The second procedure may be determined usingthe position on the ground of the first target such that a direction inwhich the aircraft flies when the first target is imaged during thesecond procedure is different to (e.g. substantially perpendicular to) adirection in which the aircraft was flying during the first procedurewhen an image of the first target was captured.

Determining the second procedure may comprise minimising overlap betweenthe first route and the second route. This tends minimise the number ofpoints on the ground that the aircraft images from the same direction.

The one or more processors may be located on-board the aircraft.

The method may further comprise acquiring, by the one or moreprocessors, a specification of a volume of airspace. The step ofdetermining the first procedure may comprises, using the specificationof the volume of airspace, determining the first procedure such that theaircraft remains with the volume of airspace during the first procedure.

The first procedure may be determined such that the number of turnsperformed by the aircraft whilst performing the first procedure isminimised.

The method may further comprise: processing the captured images todetect, within at least one image, a second target; acquiring, by theone or more processors, a position on the ground of the detected secondtarget; using the acquired specification of possible positions andorientations relative to the aircraft to which the sensor may be moved,the acquired position of the second target, and the specification of themaneuverability of the aircraft, determining, by the one or moreprocessors, a third procedure to be performed by the aircraft;performing, by the aircraft, the third procedure; and, whilst theaircraft performs the third procedure, capturing, by the sensor, a thirdset of images; wherein the third procedure comprises the aircraft movingwith respect to the second target and the sensor moving with respect tothe aircraft such that the second target is coincident with a footprintof the sensor on the ground for the entire duration of the thirdprocedure; and capturing the third set of images is performed such thatthe whole of the second target is present within each image in the thirdset.

The method may further comprise: processing the captured images todetect, within at least one image, a third target; acquiring, by the oneor more processors, a position on the ground of the detected thirdtarget; acquiring, by the one or more processors, a specification of adirection relative to the aircraft in which an exhaust of the aircraftpoints; and, using the acquired position of the third target, thespecification of the maneuverability of the aircraft, and the acquiredspecification of the direction, determining by the one or moreprocessors, a third route for the aircraft; and following, by theaircraft, the third route. The determination of the third route maycomprise minimising a duration for which the exhaust of the aircraft isdirected towards the third target.

The aircraft may comprise a payload releasably attached to the aircraft.The method may further comprise: processing the captured images todetect, within at least one image, a fourth target; acquiring, by theone or more processors, a position on the ground of the detected fourthtarget; acquiring, by the one or more processors, parameter valuesrelating to properties of the payload; acquiring, by the one or moreprocessors, parameter values relating to environmental conditions inwhich the aircraft is flying; using the acquired position of the fourthtarget, the acquired parameter values relating to properties of thepayload, and the acquired parameter values relating to environmentalconditions, determining, by the one or more processors, a position and avelocity for the aircraft; using the determined position and velocityfor the aircraft, determining, by the one or more processors, a fourthprocedure for the aircraft; performing, by the aircraft, the fourthprocedure; and, at a point in the fourth procedure at which the aircrafthas the determined position and velocity, releasing, by the aircraft,the payload. The determined position and a velocity for the aircraft maybe such that, were the aircraft to release the payload whilst located atthe determined position and travelling at the determined velocity, thepayload would land on the ground within a predetermined distance of thefourth target. The fourth procedure is such that, were the aircraft toperform the fourth procedure, at at least one instance during the fourthprocedure, the aircraft would be located at the determined position andtravelling at the determined velocity.

Capturing a set of images may comprise, for each image: acquiring, byone or more processors, a specification of a region on the ground to beimaged; measuring, by a position sensor fixedly mounted to a rigidsupport structure, a position of the position sensor; measuring, by anorientation sensor fixedly mounted to the rigid support structure, anorientation of the orientation sensor; using the measured position andorientation and using the acquired region specification, determining aposition and orientation for the sensor, the sensor being mounted to therigid support structure; controlling the aircraft and the orientation ofthe sensor on-board the aircraft such that the sensor has the determinedposition and orientation, thereby providing that a footprint of thesensor on the ground is coincident with the region on the ground to beimaged; and, when the sensor has the determined position andorientation, capturing, by the sensor, one or more images of the area ofthe ground within the sensor footprint. The rigid support structure maybe releasably coupled to the aircraft of the aircraft.

In a further aspect, the present invention provides apparatus forimaging an area of terrain, the apparatus comprising: a sensor movablymounted on-board an aircraft; one or more processors configured to:acquire a specification of possible positions and orientations relativeto the aircraft to which the sensor may be moved; acquire positionalinformation of the area of terrain; acquire a specification of themaneuverability of the aircraft; using the acquired specification ofpossible positions and orientations relative to the aircraft to whichthe sensor may be moved, the acquired positional information of the areaof terrain, and the acquired specification of the maneuverability of theaircraft, determine a first procedure and a second procedure; and meansfor controlling the aircraft to perform the first procedure and,thereafter, the second procedure. The first procedure comprises theaircraft moving with respect to the area of terrain along a first routeand the sensor moving with respect to the aircraft such that, for eachpoint in the area of terrain, that point is coincident with a footprintof the sensor on the ground for at least some time during the firstprocedure; the second procedure comprises the aircraft moving withrespect to the area of terrain along a second route and the sensormoving with respect to the aircraft such that, for each point in thearea of terrain, that point is coincident with a footprint of the sensoron the ground for at least some time during the second procedure. Thefirst route is different to the second route. The sensor is configuredto: whilst the aircraft performs the first procedure, capture a firstset of images, each image in the first set being of only part of thearea of terrain, the first set of images being such that, for everypoint in the area of terrain, that point is present within at least oneof the images in the first set; and, whilst the aircraft performs thesecond procedure, capture a second set of images, each image in thefirst set being of only part of the area of terrain, the second set ofimages being such that, for every point in the area of terrain, thatpoint is present within at least one of the images in the second set.

In a further aspect, the present invention provides an imaging methodfor imaging an area of terrain using a sensor mounted on an unmannedaircraft (e.g. an autonomous unmanned aircraft). The method is forperforming a wide area search of the area of terrain. The area ofterrain, sensor, and altitude of the aircraft may be such that the areaof terrain is too large to be wholly contained within a single imagetaken by the sensor. The method comprises: acquiring, by one or moreprocessors, a range of motion of the sensor relative to the aircraft(i.e. information that defines or specifies possible positions andorientations relative to the aircraft to which the sensor may be moved);acquiring, by the one or more processors, positional information of thearea of terrain to be imaged; acquiring, by the one or more processors,parameter values relating to the maneuverability of the aircraft; usingthe acquired range of motion of the sensor, the acquired positionalinformation of the area of terrain, and the acquired parameter valuesrelating to the maneuverability of the aircraft, determining, by the oneor more processors, a procedure to be performed by the aircraft;performing, by the aircraft, the determined procedure; and, whilst theaircraft performs the procedure, capturing, by the sensor, a set ofimages. The procedure comprises the aircraft moving with respect to thearea of terrain and the sensor moving with respect to the aircraft suchthat, for each point in the area of terrain, that point is coincidentwith a footprint of the sensor on the ground for at least some timeduring the procedure, and capturing the images is performed such that,for every point in the area of terrain, that point is present within atleast one of the captured images.

The one or more processors may be located on-board the aircraft.

The step of determining the procedure may comprise determining a routefor the aircraft to follow and determining an imaging schedule for thesensor. The step of the performing, by the aircraft, the procedure maycomprise the aircraft following the route. The step of capturing, by thesensor, the images may be performed in accordance with the determinedimaging schedule.

The method may further comprise, using the acquired range of motion ofthe sensor, the acquired positional information of the area of terrain,and the acquired parameter values relating to the maneuverability of theaircraft, determining, by the one or more processors, a furtherprocedure to be performed by the aircraft. The method may furthercomprise, sometime after the performance of the procedure by theaircraft, performing, by the aircraft, the further procedure. The methodmay further comprise, whilst the aircraft performs the furtherprocedure, capturing, by the sensor, a further set of images. Thefurther procedure may comprise the aircraft moving with respect to thearea of terrain and the sensor moving with respect to the aircraft suchthat, for each point in the area of terrain, that point is coincidentwith a footprint of the sensor on the ground for at least some timeduring the further procedure. Capturing the further set of images may beperformed such that, for every point in the area of terrain, that pointis present within at least one of the images in the further set ofimages. The route followed by the aircraft during the procedure may bedifferent to a route followed by the aircraft during the furtherprocedure. Preferably, for a region within the area of terrain, adirection in which the aircraft flies while that region is imaged duringthe procedure is substantially perpendicular to a direction in which theaircraft flies while that region is imaged during the further procedure.

The method may further comprise acquiring, by the one or moreprocessors, a specification of a volume of airspace. The step ofdetermining the procedure may comprise using the specification of thevolume of airspace. The procedure may be such that the aircraft remainswith the volume of airspace during the performance of the procedure.

The procedure may be determined such that the number of turns performedby the aircraft whilst performing the procedure is minimised.

The method may further comprise: for each image in the set, determining,by one or more processors on-board the aircraft, a set of properties ofthat image; performing, by the one or more processors, a targetdetection process on the set of images to detect one or more firsttargets within the set of images; for a each first target, determining,by the one or more processors, a set of properties of that first target;transmitting, by a transmitter on-board the aircraft, for use by anentity remote from the aircraft, the determined image properties;transmitting, by the transmitter, for use by the entity, the determinedfirst target properties; by the entity remote from the aircraft, usingthe received image properties and first target properties, identifying aregion of interest on the ground; sending, from the entity to theaircraft, a request for image data relating to the region of interest;receiving, by a receiver on-board the aircraft, the request; and, inresponse to receiving the request, transmitting, by the transmitter, foruse by the entity, the image data relating to the determined region ofinterest.

The method may further comprise processing the captured images todetect, within at least one image, a second target; acquiring, by theone or more processors, a position on the ground of the detected secondtarget; using the acquired range of motion of the sensor, the acquiredposition of the second target, and the acquired parameter valuesrelating to the maneuverability of the aircraft, determining, by the oneor more processors, a second further procedure to be performed by theaircraft; performing, by the aircraft, the second further procedure;and, whilst the aircraft performs the second further procedure,capturing, by the sensor, a second further set of images. The secondfurther procedure may comprise the aircraft moving with respect to thetarget and the sensor moving with respect to the aircraft such that thesecond target is coincident with a footprint of the sensor on the groundfor the entire duration of the second further procedure. Capturing thesecond further set of images may be performed such that the whole of thesecond target is present within each image in the second further set.

The method may further comprise: processing the captured images todetect, within at least one image, a third target; acquiring, by the oneor more processors, a position on the ground of the detected thirdtarget; acquiring, by the one or more processors, a specification of adirection relative to the aircraft in which an exhaust of the aircraftpoints; using the acquired position of the third target, the acquiredparameter values relating to the maneuverability of the aircraft, andthe acquired specification of the direction, determining by the one ormore processors, a route for the aircraft; and following, by theaircraft, the determined route. The determination of the route maycomprise minimising a duration for which the exhaust of the aircraft isdirected towards the third target.

The aircraft may comprise a payload releasably attached to an aircraftand the method may further comprise: processing the captured images todetect, within at least one image, a fourth target; acquiring, by theone or more processors, a position on the ground of the detected fourthtarget; acquiring, by the one or more processors, parameter valuesrelating to properties of the payload; acquiring, by the one or moreprocessors, parameter values relating to environmental conditions inwhich the aircraft is flying; using the acquired position of the fourthtarget, the acquired parameter values relating to properties of thepayload, and the acquired parameter values relating to environmentalconditions, determining, by the one or more processors, a position and avelocity for the aircraft; using the determined position and velocityfor the aircraft, determining, by the one or more processors, a thirdfurther procedure for the aircraft; performing, by the aircraft, thethird further procedure; and, at a point in the third further procedurethat the aircraft has the determined position and velocity, releasing,by the aircraft, the payload. The determined position and a velocity forthe aircraft may be such that, were the aircraft to release the payloadwhilst located at the determined position and travelling at thedetermined velocity, the payload would land on the ground within apredetermined distance of the fourth target. The third further proceduremay such that, were the aircraft to perform the third further procedure,at at least one instance during the third further procedure, theaircraft would be located at the determined position and travelling atthe determined velocity.

The step of capturing the set of images may comprise, for each image:acquiring, by one or more processors, a specification of a region on theground to be imaged; measuring, by a position sensor fixedly mounted toa rigid support structure, its position; measuring, by an orientationsensor fixedly mounted to the rigid support structure, its orientation;using the measured position and orientation and using the acquiredregion specification, determining a position and orientation for thesensor, the sensor being mounted to the rigid support structure;controlling the aircraft and the orientation of the sensor on-board theaircraft such that the sensor has the determined position andorientation, thereby providing that a footprint of the sensor on theground is coincident with the region on the ground to be imaged; and,when the sensor has the determined position and orientation, capturing,by the sensor, one or more images of the area of the ground within thesensor footprint.

In a further aspect, the present invention provides apparatus forimaging an area of terrain, the apparatus comprising: a sensor mountedon-board an aircraft (e.g. an autonomous unmanned aircraft); one or moreprocessors (e.g. located on-board the aircraft) configured to: acquire arange of motion of the sensor relative to the aircraft (i.e. informationthat defines or specifies possible positions and orientations relativeto the aircraft to which the sensor may be moved); acquire positionalinformation of the area of terrain; acquire parameter values relating tothe maneuverability of the aircraft; using the acquired range of motionof the sensor, the acquired specification of the area of terrain, andthe acquired parameter values relating to the maneuverability of theaircraft, determine a procedure to be performed by the aircraft; andmeans for controlling the aircraft to perform the determined procedure.The sensor is configured to, whilst the aircraft performs the procedure,capture a set of images. The procedure comprises the aircraft movingwith respect to the area of terrain and the sensor moving with respectto the aircraft such that, for each point in the area of terrain, thatpoint is coincident with a footprint of the sensor on the ground for atleast some time during the procedure. Capturing the images is such that,for every point in the area of terrain, that point is present within atleast one of the captured images.

In a further aspect, the present invention provides a program orplurality of programs arranged such that when executed by a computersystem or one or more processors it/they cause the computer system orthe one or more processors to operate in accordance with the method ofany of the above aspects.

In a further aspect, the present invention provides a machine readablestorage medium storing a program or at least one of the plurality ofprograms according to the preceding aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration (not to scale) showing a scenario;

FIG. 2 is a schematic illustration (not to scale) of an aircraft;

FIG. 3 is a schematic illustration (not to scale) of a sensor module;

FIG. 4 is a process flow chart showing certain steps of a process inwhich an imaging process is performed;

FIG. 5 is a schematic illustration (not to scale) showing the aircraftfollowing a flight path defined by the waypoints;

FIG. 6 is a process flow chart showing certain steps in a firstembodiment of the imaging process;

FIG. 7 is a schematic illustration (not to scale) of the aircraftperforming a wide area search;

FIG. 8 is a process flow chart showing certain steps in a secondembodiment of the imaging process;

FIG. 9 is a schematic illustration (not to scale) showing the aircraftperforming a feature following process;

FIG. 10 is a process flow chart showing certain steps in a thirdembodiment of the imaging process;

FIG. 11 is a schematic illustration (not to scale) of the aircraftperforming a surveillance operation;

FIG. 12 is a process flow chart showing certain steps of an imageprocessing method; and

FIG. 13 is a process flow chart showing certain steps of a payloaddelivery process.

DETAILED DESCRIPTION

FIG. 1 is a schematic illustration (not to scale) showing an examplescenario 1 in which embodiments of an imaging process is to beimplemented.

The scenario 1 comprises an aircraft 2 and a ground station 4.

The aircraft 2 is described in more detail later below with reference toFIG. 2.

In the scenario 1, as described in more detail later below as theaircraft 2 is airborne, systems on board the aircraft 2 capture highresolution visible band images of an area on the ground 8. Processedimage data is then sent from the aircraft 2 to the ground station 4 viaa wireless communications link 6.

In the scenario 1, the ground station 4 is located on the ground 8 andis remote from the aircraft 2.

FIG. 2 is a schematic illustration (not to scale) of the aircraft 2. Theimaging process performed by the aircraft 2 in this scenario 1 will bedescribed in more detail later below with reference FIG. 5.

The aircraft 2 is an unmanned aircraft. The aircraft 2 comprises asensor module 10, a processor 12, a storage module 14, a plurality ofaircraft subsystems (which are hereinafter collectively referred to as“the aircraft subsystems” and indicated in FIG. 1 by a single box andthe reference numeral 16), a transceiver 18, and a payload 19.

The sensor module 10 is described in more detail later below withreference to FIG. 3. In this embodiment, the sensor module 10 isconnected to the processor 12 such that information may be sent betweenthe sensor module 10 and the processor 12.

In this embodiment, the processor 12 is configured to processinformation received by it as described in more detail later below. Inaddition to being connected to the sensor module 10, the processor 12 isconnected to the storage module 14 such that information may be sentfrom the processor 12 to the storage module 14 (for storage by thestorage module 14) and such that information stored by the storagemodule 14 may be acquired by the processor 12. The processor 12 isfurther connected to the aircraft subsystems 16 such that informationmay be sent between the processor and the aircraft subsystems 16. Theprocessor 12 is further connected to the transceiver 18 such thatinformation may be sent between the processor 12 and the transceiver 18.The processor 12 is further connected to the payload 19 such thatinformation may be sent between the processor 12 and the payload 19.

In this embodiment, the storage module 14 is configured to storeinformation received from the processor 12.

In this embodiment, the aircraft subsystems 16 include, but are notlimited to, a propulsion system of the aircraft 2, a power system of theaircraft 2, a fuel system of the aircraft 2, and a navigation system ofthe aircraft 2. The propulsion system may, for example, include primaryand auxiliary propulsion units for generating thrust and/or lift. Inthis embodiment, the propulsion system includes sensing apparatus fromwhich data relating to the propulsion of the aircraft 2 (e.g. theaircraft's speed) may be acquired. The power system may compriseelectrical power and power distribution systems for providing electricalpower to other aircraft systems. In this embodiment, the power systemincludes sensing apparatus from which data relating to a state oroperations of the power system may be acquired. The fuel system maycomprise fuel storage (such as fuel tanks), monitoring (such as fuellevel, temperature and/or pressure sensors), and distribution systems(such as supply lines). In this embodiment, the fuel system includessensing apparatus from which data relating to the fuel system may beacquired. In this embodiment, the navigation system includes sensingapparatus for determining, at least, a global position of the aircraft 2(e.g. a Global Positioning System receiver), an altitude of the aircraft2, and a heading of the aircraft 2.

In this embodiment, the transceiver 18 is configured to receiveinformation from an entity that is remote from the aircraft 2 and relaythat information to the processor 12. Also, the transceiver 18 isconfigured to transmit, for use by an entity that is remote from theaircraft 2, information received by the transceiver 18 from theprocessor 12.

In this embodiment, the payload 19 may be any appropriate type of loador cargo, such as a container containing food supplies or equipment. Thepayload 19 may be a lethal effector or a non-lethal effector.

In this embodiment, the processor 12 may operate so as to release thepayload 19 from the aircraft in flight so that the payload 19 is free tomove away from the aircraft 2. In some embodiments, the payload 19 is a“dumb” payload. In some embodiments, the payload 19 is a steered payloadand may be controlled, e.g. by the processor 12, so as to changedirection after it has been released from the aircraft 2. In someembodiments, the payload 19 is a guided payload. In some embodiments,the payload 19 may include a parachute which may be deployed after thepayload 19 is released from the aircraft 2 so as to slow the decent ofthe payload 19 from the aircraft 2 to the ground 8.

FIG. 3 is a schematic illustration (not to scale) showing the sensormodule 10. In this embodiment, the sensor module 10 is detachable fromthe aircraft fuselage and may, e.g., be replaced by a further sensormodule comprising a different type of sensor.

In this embodiment, the sensor module 10 comprises a camera 20, aninterface module 22, a position and orientation module 24, a rigidsupport structure 26, and a moveable turret 28.

In this embodiment, the camera 20 is a visible light detecting cameraconfigured to capture visible light images as described in more detaillater below. The camera 20 is mounted to the turret 28 such that, bymoving or steering the turret 28, the position and orientation of thecamera 20 relative to the support structure 26 may be changed. In otherwords, by steering the turret 28, the facing of the camera 20 may bechanged. In this embodiment, the camera 20 is connected to the interfacemodule 22 such that, as described in more detail later below, a controlsignal may be sent from the interface module 22 to the camera 20 andsuch that image data may be sent from the camera 20 to the interfacemodule 22.

In this embodiment, the interface module 22 is configured to processinformation received by the interface module 22 as described in moredetail later below. In this embodiment, the interface module 22 ismounted to the support structure 26 such that the interface module 22has a fixed position and orientation relative to the support structure26. In addition to being connected to the camera 20, the interfacemodule 22 is connected to the processor 12 such that information may besent between the interface module 22 and the processor 12. The interfacemodule 22 is also connected to the position and orientation module 24such that information may be sent between the interface module 22 andthe position and orientation module 24.

In this embodiment, the position and orientation module 24 comprises aposition sensor 30 and an orientation sensor 32. The position sensor 30is configured to measure a global position of the position sensor 30.The position sensor 30 may, for example, include a GPS receiver. Theorientation sensor 32 is configured to measure an orientation of theorientation sensor 32. The orientation sensor may, for example, includea compass. In this embodiment, the position and orientation module 24 ismounted to the support structure 26 such that the position sensor 30 andan orientation sensor 32 each have a fixed position and orientationrelative to the support structure 26.

In this embodiment, the turret 28 is a steerable sensor turret. Theturret 28 is attached between the camera 20 and the support structure 26such that, by steering the turret 28 such that, by steering the turret28, the facing of the camera 20 with relative to the support structure26 may be altered. In this embodiment, the turret 28 is connected to theinterface module 24 such that a control signal (i.e. a signal forsteering the turret 28) may be sent from the interface module 24 to theturret 28. The turret 28 is configured to operate in accordance with areceived control signal.

In this embodiment the turret 28 is a gimballed sensor turret configuredto allow for rotation of the camera 20 around multiple (e.g. two orthree) orthogonal axes with respect to the support structure 26. Forexample, the turret 28 may comprise three gimbals coupled together suchthat the pivot axes of the gimbals are orthogonal to one another. Inthis embodiment, the rigid support structure 26 is fixedly attached tothe fuselage of the aircraft 2. The support structure 26 is resistant tobending and flexing as the aircraft 2 flies.

In some embodiments, the turret 28 is attached directly to the fuselageof the aircraft 2, i.e. the rigid support structure 26 may be omitted.

In operation, as the aircraft 2 flies in the proximity of an area ofterrain or a terrain feature, the camera 20 captures high resolutionvisible band images of that area of terrain or terrain feature, asdescribed in more details later below. The area of terrain or terrainfeature that is imaged using the camera 20 is hereinafter referred to asthe “imaging target”. Data corresponding to the images captured by thecamera 20 is sent from the camera 20 to the processor 12. The processor12 is used to perform an image processing method on the received data.

In this embodiment, during the image processing method, processed datais sent from the processor 12 to the storage module 14 where it isstored, as described in more detail later below. Also, processed data issent from the processor 12 to the transceiver 18 where it is transmittedto the ground station 4.

FIG. 4 is a process flow chart showing certain steps of a processperformed by the entities in the scenario 1.

At step s2, a specification of the imaging target that is to be imagedis provided to the aircraft 2. In this embodiment, the specification ofthe imaging target is stored in the storage module 14. In thisembodiment, the specification of the imaging target is loaded onto theaircraft 2 prior to the aircraft 2 taking off. However, in otherembodiments, the specification of the imaging target may be transmittedto the aircraft 2, e.g. from the ground station 4, while the aircraft 2is airborne.

The imaging target may, for example, be specified using globalcoordinates (i.e. latitudes and longitudes).

At step s4, a specification of a volume of airspace in which theaircraft 2 is permitted to fly whilst imaging the imaging target isprovided to the aircraft 2. In this embodiment, the specification of thevolume of airspace is stored in the storage module 14. In thisembodiment, the specification of the volume of airspace is loaded ontothe aircraft 2 prior to the aircraft 2 taking off. However, in otherembodiments, the specification of the volume of airspace may betransmitted to the aircraft 2, e.g. from the ground station 4, while theaircraft 2 is airborne.

The volume of airspace may, for example, be specified using globalcoordinates and altitudes.

At step s6, a sequence of waypoints is provided to the aircraft 2. Inthis embodiment, a waypoint is a point in the air that the aircraft 2 isto fly through, or within a pre-determined distance of. The sequence ofwaypoints define a flight-path for the aircraft from the aircraft'stake-off point, to a point within, or within a predetermined distanceof, the volume of airspace specified at step s4.

In this embodiment, the specification of the sequence of waypoints isstored in the storage module 14. In this embodiment, the specificationof the sequence of waypoints is loaded onto the aircraft 2 prior to theaircraft 2 taking off. However, in other embodiments, the specificationof the sequence of waypoints may be transmitted to the aircraft 2, e.g.from the ground station 4, while the aircraft 2 is airborne.

The sequence of waypoints may, for example, be specified using globalcoordinates and altitudes. In some embodiment, one or more of thewaypoints may be a different type of point that can be used to define aroute for the aircraft 2. For example, in some embodiments, a waypointis a point on the ground over which the aircraft 2 is to fly.

In this embodiment, steps s2 to s6 are performed prior to the aircraft 2taking off. However, in other embodiments the data corresponding to oneor more of the waypoints, the imaging target, and/or the volume ofairspace may be provided to the aircraft 2 at a different time, forexample when the aircraft 2 is airborne (in which case, this data may beprovided to the aircraft 2 from the ground station 4 via thecommunications link 6). Thus, tasks can advantageously be uploadedwhilst the aircraft 2 is airborne. Furthermore, tasks for the aircraft 2can updated, modified, cancelled, replaced, added to etc. whilst theaircraft 2 is airborne.

At step s8, the aircraft 2 takes-off from the ground station 4.

At step s10, the aircraft 2 follows the flight path defined by thesequence of waypoints stored in the storage module 14 until the aircraft2 enters the volume of airspace. In this embodiment, the aircraft 2 isunmanned and autonomous.

FIG. 5 is a schematic illustration (not to scale) showing the sequenceof waypoints 34 followed by the aircraft 2. In this embodiment, theflight path 36 defined by the waypoints 34 is followed by the aircraft 2until the aircraft 2 enters the volume of airspace 38. FIG. 5 furthershows the imaging target 40.

At step s12, upon entering the volume of airspace 38, the aircraftperforms an imaging process to capture images of the imaging target 40.In this embodiment, the sensor module 10 captures images of the imagingtarget 40.

A first embodiment of the imaging process is described in more detaillater below with reference to FIGS. 6 and 7.

A second embodiment of the imaging process is described in more detaillater below with reference to FIGS. 8 and 9.

A third embodiment of the imaging process is described in more detaillater below with reference to FIGS. 10 and 11.

At step s14, the sensor module 10 sends the captured images to theprocessor 12.

At step s16 the processor 12 performs an image processing method on thereceived images.

An embodiment of an image processing method is described in more detaillater below with reference to FIG. 12.

At step s18, the aircraft 2 completes its journey, for example, byreturning to its launch-site (e.g. the ground station 4), for example,by following the flight path 36.

Thus, a process in which an imaging process is performed is provided.

What will now be described is a first embodiment of an imaging processperformed at step s12.

In this first embodiment, the imaging target 40 is a relatively largedefined area of terrain. In the first embodiment, the imaging of theimaging target 40 comprises conducting a “wide area search” of theimaging target 40. The terminology “wide area search” is used herein torefer to the reconnaissance of the imaging target 40 that includestaking images of the imaging target 40 such that each point in theimaging target 40 is contained in at least one of those images. In thisembodiment, the relatively large defined area of terrain is such thatthe entirety of the area of terrain cannot be captured in a single imagetaken by the camera 20. In this embodiment, a wide area search furthercomprises processing the captured images to detect targets of interestwithin those images.

FIG. 6 is a process flow chart showing certain steps in the firstembodiment of the imaging process.

At step s20, the processor 12 acquires the specification for the imagingtarget 40 and the specification for the volume of airspace 38 that arestored in the storage module 14.

In this embodiment, the specification of the imaging target 40 includesglobal positions of points along the border of the imaging target 40,i.e. a definition of the border of the large defined area of terrainthat is to be imaged.

At step s22, the processor 12 acquires current performance parametervalues for the aircraft 2 from the aircraft subsystems 16. Examples ofappropriate aircraft performance parameter values include, but are notlimited to, velocities at which the aircraft 2 is capable of travelling,altitudes at which the aircraft 2 is capable of travelling, and aturning radius for the aircraft 2

At step s24, the processor 12 acquires performance parameter values forthe sensor module 10 from the interface module 22. Examples ofappropriate performance parameter values for the sensor module 10include, but are not limited to, the frequency with which the camera 20can capture images, a range of motion, or range of travel, of the turret28 relative to the support structure 26, and a maximum speed at whichthe turret 28 may move.

The range of motion of the turret 28 may specify a distance (linearand/or angular), relative to the support structure 26, that the moveableturret 28 may travel while properly attached to the support structure26. The range of motion of the turret 28 may specify, for each of one ormore axes (e.g. multiple orthogonal axes), an angular distance aboutthat axis relative to the support structure 26 that the turret 28 iscapable of moving. The range of motion of the turret 28 may define therange of possible positions and facings relative to the supportstructure 26 that the camera 20 may occupy by operation of the turret28.

In some embodiments, the aircraft 2 is required to perform the wide areasearch of the imaging target 40 within a pre-specified amount of time.In such embodiments, the processor 12 may also acquire a specificationof this time period.

At step s26, using some or all of the information acquired at steps s20,s22, and s24, the processor 12 determines a route, hereinafter referredto as the “first route”, within the volume of airspace 38 for theaircraft 2. The determined first route is such that, were the aircraft 2to follow that route, the sensor module 10 would be capable of capturingimages of the imaging target 40 such that each point in the imagingtarget 40 is contained within at least one of those images.

Further information about the first route determined by the processor 12at step s26 is described in more detail later below with reference toFIG. 7.

At step s28, the processor 12 determines an imaging schedule,hereinafter referred to as the “first imaging schedule”, for the sensormodule 10. In this embodiment, the first imaging schedule specifies asequence of points along the first route and, for each of those points,one or more regions within the imaging target 40 of which the camera 20is to capture an image.

In this embodiment, the first imaging schedule is such that, were thecamera 20 to capture images in accordance with that imaging schedule,each point on the ground 8 within the imaging target 40 would becontained within at least one of the captured images (i.e. a wide areasearch of the imaging target 40 would be performed).

Each of the sequence of points along the first route may be specified,for example, by an aircraft position (e.g. as GPS coordinates and analtitude). Each of the regions within the imaging target 40 that thecamera 20 is to capture an image of may be specified by GPS coordinatesfor that region.

Further information about the first imaging schedule is described inmore detail later below with reference to FIG. 7.

At step s30, the processor 12 sends the first imaging schedule to theinterface module 22 of the sensor module 10.

At step s32, the aircraft 2 is controlled (e.g. by the processor 12) soas to follow the first route.

At step s34, as the aircraft 2 follows the first route, the interfacemodule 22 acquires position and orientation measurements from theposition and orientation module 24. In particular, the interface module22 acquires position measurements from the position sensor 30 andorientation measurements from the orientation sensor 32.

In this embodiment, the position and orientation module 24 is fixed tothe support structure 26. Also, the turret 28 is fixed to the supportstructure 26. Thus, using the acquired position and orientationmeasurements, and using a known positional relationship between theturret 28 and the position and orientation module 24, and using theknown orientation of the turret 28 relative to the support structure 26,the interface module 22 is able to determine a current position andorientation for the camera 20.

At step s36, using the determined current position and orientation ofthe camera 20 and using the first imaging schedule received from theprocessor 12, as the aircraft 2 follows the first route, the interfacemodule 22 controls the turret 28 and the camera 20 so as to captureimages in accordance with the first imaging schedule.

For example, a step in the first imaging schedule may specify a regionwithin the imaging target 40 of which an image is to be captured. Usingspecification of that region, the interface module 22 determines aposition and orientation for the camera 20 that would provide that thespecified region is wholly located in the camera's footprint on theground 8. When that step in the first imaging schedule is reached, theinterface module 22 controls the turret 28 so that the camera 20 has thedetermined position and orientation. Once the camera 20 has the desiredposition and orientation, the interface module 22 controls the camera 20to capture one or more images of the specified region.

Further information about the capturing of images performed at step s36is described in more detail later below with reference to FIG. 7.

At step s38, the camera 20 sends the captured images to the interfacemodule 22.

At step s40, the interface module 22 processes the received images so asto convert those images into a predetermined format (e.g. a standardisedformat) that is usable by the processor 12.

After step s40, the method proceeds back to step s14 of FIG. 4, at whichpoint the interface module 22 sends the converted images to theprocessor 12.

Thus, a first embodiment of the imaging process is provided.

FIG. 7 is a schematic illustration (not to scale) of the aircraft 2performing a wide area search of the imaging target 40, as describedabove with reference to FIG. 6.

In FIG. 7, the first route is indicated by the reference numeral 42. Thedirection of travel of the aircraft 2 along the first route 42 isindicated in FIG. 7 by arrow heads placed along the first route 42.

Also, in FIG. 7, the region of the ground 8 that is able to be imaged bythe camera 20 at a particular time-step is indicated by the referencenumeral 44. This region is referred to herein as the “camera footprint”.

In this embodiment, the size of the camera footprint 44 on the ground 8at a particular time-step is dependent on the position and orientationof the camera 20 relative to the aircraft 2 (which may be controlled bycontrolling the turret 28), the position and orientation of the aircraft2 (including the altitude of the aircraft 2 above the ground 8), and thesurface relief of the ground 8 (information relating to which may, forexample, be loaded onto the aircraft 2 prior to take-off). The size ofthe camera footprint 44 on the ground 8 at a particular time-step may bedetermined by the processor 12.

In this embodiment, when viewed from above, the first route 42 issubstantially S-shaped. The first route 42 comprises three parallelstraight sections that are connected together by curved sections. Inother embodiments, the first route 42 may have a different shape, forexample, the first route 42 may include a different number of straightsections and curved sections.

As the aircraft 2 flies along the straight sections of the first route42, the camera footprint 44 is moved over the imaging target 40 in thedirection of travel of the aircraft 2 (as indicated in FIG. 7 by anarrow and the reference numeral 46). Also as the aircraft 2 flies alongthe straight sections of the first route 42, the turret 28 may becontrolled such that the camera footprint 44 is swept back and forth ina direction that is perpendicular to the direction of travel of theaircraft 2 (as indicated in FIG. 7 by arrows the reference numerals 48).In this embodiment, the camera 20 is controlled so as to capture imagesof the imaging target 40 as the aircraft 2 flies along the straightsections of the first route 42. Thus, as the aircraft flies along thestraight section of the first route 42 a strip of the imaging target 40is imaged. In this embodiment, for each straight section of the firstroute 42, a length of the strip of the imaging target 40 that is imagedas the aircraft 2 flies along that straight sections is greater than orequal to the entire length of the imaging target 40.

In this embodiment, the distance between a straight section of the firstroute 42 and a subsequent straight section of the first route 42 is suchthat the strip of the imaging target 40 that is imaged while theaircraft 2 flies along the straight section overlaps at least to someextent with the strip of the imaging target 40 that is imaged while theaircraft 2 flies along the subsequent straight section.

In this embodiment, the number of straight sections is such that theentirety of the imaging target 40 is imaged during the straight sectionsof the first route 42.

In this embodiment, the curved sections of the first route 42 aresections at which the aircraft 2 turns, i.e. changes direction, betweenstraight sections of the first route 42. Preferably, the first route 42is determined such that the number of turns the aircraft 2 has to makeis minimised. In this embodiment, the radius of each of the curvedsections, which is denoted in FIG. 7 by double headed arrows and thereference numeral 49, is dependent upon the minimum turn radius of theaircraft 2. In particular, for each curved section of the first route42, the radius 49 of that curved section is greater than or equal to theminimum turn radius of the aircraft 2.

By flying along the first route 42 and by controlling the turret 28, thecamera footprint 44 is moved over the entirety of the imaging target 40.Thus, each point within the imaging target 40 is contained within atleast one image taken by the camera 20.

In other embodiments, the first route 42 has a different shape to thatdescribed above. Also, in other embodiments, the turret 28 is controlledso as to move the camera footprint 44 in a different way to thatdescribed above, while still providing that each point within theimaging target 40 is contained within at least one image captured by thecamera 20 as the aircraft 2 follows the first route 42.

In some embodiments, after following the first route 42 and imaging theentirety of the imaging target 40, the processor 12 calculates a furtherfirst route and a further first imaging schedule. The further firstroute and the further first imaging schedule may be such that, were theaircraft 2 to follow the further first route and capture images inaccordance with the further first imaging schedule, each point on theground 8 within the imaging target 40 would be contained within at leastone of the captured images (i.e. a further wide area search of theimaging target 40 would be performed). The images captured during thefirst imaging schedule may be registered with those captured during thefurther first imaging schedule. For example, the images captured duringthe first imaging schedule and the images captured during the furtherfirst imaging schedule may be transformed into a global coordinatesystem. An advantage provided by performing more than one wide areasearch of the imaging target 40 is that errors in determinedgeolocations of detected targets tend to be reduced. In particular, whendetermining a geolocation of a detected target from an image (e.g. asdescribed in more detail later below), the uncertainty associated withthat determined geolocation tends to be largest in the direction oftravel of the aircraft 2 when that image was taken. Using more than oneimage to determine a geolocation of a detected target tends toadvantageously decrease the associated uncertainty. Preferably, for eachimage used to determine a geolocation of a target, the direction thatthe aircraft 2 was travelling when that image was taken is different(for example, preferably perpendicular) to the direction that theaircraft 2 was travelling when each of the other images used todetermine the geolocation of that target was taken. This may be providedby calculating the further first route in such a way that it isdifferent to the first route 42. For example, the further first routemay be determined using a criterion that an overlap between the furtherfirst and the first route is minimised.

Advantageously, uncertainty associated with a geolocation of a point orregion tends to be greatly reduced if the direction in which theaircraft flies while that region is imaged during the first imagingschedule is substantially perpendicular to a direction in which theaircraft flies while that region is imaged during the further firstimaging schedule. Thus, in embodiments in which a target is detected inthe images captured during the first imaging schedule, the uncertaintyassociated with the geolocation of that target tends to be greatlyreduced if the direction in which the aircraft flies while that targetis imaged during the first imaging schedule is substantiallyperpendicular to a direction in which the aircraft flies while thattarget is imaged during the further first imaging schedule. Thus, thefurther first route may be determined using the position of the targetdetermined from the images captured during the first imaging schedule,such that the further first route is perpendicular to the first route atthe points on those routes at which the target is imaged.

What will now be described is a second embodiment of an imaging processperformed at step s12.

In this second embodiment, the imaging target 40 is a predefinedelongate terrain feature such as a road, a river, or a canal. Forconvenience, the imaging target 40 may be considered to be a linearterrain feature. In other embodiments, the imaging target is a differentlinear target such as a border of a country or man-defined feature. Inother embodiments, the imaging target 40 is a line along the grounddefined by a human such as an operator of the aircraft 2. In the secondembodiment, the imaging of the imaging target 40 comprises performing a“feature following” process on the imaging target 40. The terminology“feature following” is used herein to refer to the imaging of anelongate imaging target 40 along its entire length. In this embodiment,a feature following process is a process comprising taking images of theimaging target 40, such that each point along the entire length of theelongate imaging target 40 is contained in at least one of those images.In this embodiment, a feature following process further comprisesprocessing the captured images to detect targets of interest withinthose images.

In this embodiment, the imaging target 40 upon which the aircraft 2 isto perform the feature following process is a pre-specified feature, aspecification of which is uploaded into the storage module 14 prior tothe aircraft 2 taking off. However, in other embodiments, the imagingtarget 40 upon which the aircraft 2 is to perform the feature followingprocess is a linear target that has been previously detected, forexample, by performing the wide area search process (as described inmore details above with reference to FIGS. 6 and 7).

FIG. 8 is a process flow chart showing certain steps in the secondembodiment of the imaging process.

At step s42, the processor 12 acquires the specification for the imagingtarget 40 and the specification for the volume of airspace 38 that arestored in the storage module 14.

In this embodiment, the specification of the imaging target 40 includesglobal positions of points along the length of the linear imaging target40.

At step s44, the processor 12 acquires current performance parametervalues for the aircraft 2 from the aircraft subsystems 16. Examples ofappropriate aircraft performance parameter values include, but are notlimited to, velocities at which the aircraft 2 is capable of travelling,altitudes at which the aircraft 2 is capable of travelling, and aturning radius for the aircraft 2

At step s46, the processor 12 acquires performance parameter values forthe sensor module 10 from the interface module 22. Examples ofappropriate performance parameter values for the sensor module 10include, but are not limited to, the frequency with which the camera 20can capture images, a range of motion of the turret 28, and a maximumspeed at which the turret 28 may move.

At step s48, using some or all of the information acquired at steps s42,s44, and s46, the processor 12 determines a route, hereinafter referredto as the “second route”, within the volume of airspace 38 for theaircraft 2. The determined second route is such that, were the aircraft2 to follow that route, the sensor module 10 would be capable ofcapturing images of the imaging target 40 such that each point along thelength of the linear imaging target 40 is contained within at least oneof those images. In this embodiment, the second route is such that, werethe aircraft 2 to follow that route, the aircraft 2 would “follow” theimaging target 40 along its path.

Further information about the second route determined by the processor12 at step s48 is described in more detail later below with reference toFIG. 9.

At step s50, the processor 12 determines an imaging schedule,hereinafter referred to as the “second imaging schedule”, for the sensormodule 10. In this embodiment, the second imaging schedule specifies asequence of points along the second route and, for each of those points,a point along the linear imaging target 40 upon which the footprint ofthe camera 20 on the ground 8 is to be centred.

In this embodiment, the second imaging schedule is such that, were thecamera 20 to capture images in accordance with that imaging schedule,each point along the linear the imaging target 40 would be containedwithin at least one of the captured images.

Each of the sequence of points along the second route may be specified,for example, by an aircraft position (e.g. as GPS coordinates and analtitude). Each of the points along the length of the linear imagingtarget 40 upon which the camera footprint is to be centred may bespecified by GPS coordinates for that region.

Further information about the second imaging schedule is described inmore detail later below with reference to FIG. 9.

At step s52, the processor 12 sends the second imaging schedule to theinterface module 22 of the sensor module 10.

At step s54, the aircraft 2 is controlled (e.g. by the processor 12) soas to follow the second route.

At step s56, as the aircraft 2 follows the second route, the interfacemodule 22 acquires position and orientation measurements from theposition and orientation module 24. In particular, the interface module22 acquires position measurements from the position sensor 30 andorientation measurements from the orientation sensor 32.

In this embodiment, the position and orientation module 24 is fixed tothe support structure 26. Also, the turret 28 is fixed to the supportstructure 26. Thus, using the acquired position and orientationmeasurements, and using a known positional relationship between theturret 28 and the position and orientation module 24, and using theknown orientation of the turret 28 relative to the support structure 26,the interface module 22 is able to determine a current position andorientation for the camera 20.

At step s58, using the determined current position and orientation ofthe camera 20 and using the second imaging schedule received from theprocessor 12, as the aircraft 2 follows the second route, the interfacemodule 22 controls the turret 28 and the camera 20 so as to captureimages in accordance with the second imaging schedule.

For example, a step in the second imaging schedule may specify a pointalong the linear imaging target 40 upon which the footprint of thecamera 20 on the ground 8 is to be centred. Using the specification ofthat point, the interface module 22 determines a position an orientationfor the camera 20 that would provide that the footprint of the camera 20on the ground 8 is centred on that specified point. When that step inthe second imaging schedule is reached, the interface module 22 controlsthe turret 28 so that the camera 20 has the determined position andorientation. Once the camera 20 has the desired position andorientation, the interface module 22 controls the camera 20 to capturean image of the imaging target 40. The captured image is centred on thespecified point along the linear imaging target 40.

In this embodiment, an image of the imaging target 40 captured at theith step of the second imaging schedule overlaps at least to some extentwith an image of the imaging target 40 captured at the (i+1)th step ofthe second imaging schedule (if such an image is taken). Thus, eachpoint along the length of the imaging feature 40 is contained in atleast one image captured by the aircraft 2 during the feature followingprocess.

Further information about the capturing of images performed at step s58is described in more detail later below with reference to FIG. 9.

At step s60, the camera 20 sends the captured images to the interfacemodule 22.

At step s62, the interface module 22 processes the received images so asto convert those images into the predetermined format that is usable bythe processor 12.

After step s62, the method proceeds back to step s14 of FIG. 4, at whichpoint the interface module 22 sends the converted images to theprocessor 12.

Thus, a second embodiment of the imaging process is provided.

FIG. 9 is a schematic illustration (not to scale) showing a top-downview of the aircraft 2 performing a feature following process to imagethe linear imaging target 40, as described above with reference to FIG.8.

In this embodiment, the volume of airspace 38 in which the aircraft 2 ispermitted to fly during the feature following process is defined withrespect to the linear imaging feature 40. For example, in embodiments inwhich the linear imaging target 40 is a border of a country, theaircraft 2 may only be permitted to fly in the airspace above one sideof that linear feature.

In FIG. 9, the second route is indicated by the reference numeral 50.The direction of travel of the aircraft 2 along the second route 50 isindicated in FIG. 9 by arrow heads placed along the second route 50.

As in FIG. 7, in FIG. 9, the camera footprint (i.e. the ground 8 that isable to be imaged by the camera 20 at a particular time-step) isindicated by the reference numeral 44.

In this embodiment, the size of the camera footprint 44 on the ground 8at a particular time-step is dependent on the position and orientationof the camera 20 relative to the aircraft 2 (which may be controlled bycontrolling the turret 28), the position and orientation of the aircraft2 (including the altitude of the aircraft 2 above the ground 8), and thesurface relief of the ground 8 (information relating to which may, forexample, be loaded onto the aircraft 2 prior to take-off). The size ofthe camera footprint 44 on the ground 8 at a particular time-step may bedetermined by the processor 12.

In this embodiment, the imaging target 40 is a linear feature. In thisembodiment, when viewed from above, the shape of the second route 50 issubstantially the same as that of the imaging target 40. In effect, theaircraft 2 “follows” the path of the linear imaging target 40.

In this embodiment, the second imaging schedule specifies a sequence ofpoints (indicated by Xs in FIG. 9) along the linear imaging feature 40.The points X are points on the imaging target 40 upon which the camerafootprint 44 is to be centred when images are captured. In thisembodiment, the sequence of points X are determined by the interfacemodule 22 dependent inter alia upon the size of the camera footprint 44on the ground 8 such that, when the aircraft 2 follows the second route50 and implements the second imaging schedule, each and every pointalong the entire length of the imaging target 40 is contained within atleast one of the captured images.

In this embodiment, as the aircraft 2 follows the second route 50, thecamera footprint 44 is moved along the length of the imaging target 40,and the turret 28 may be controlled, such that the camera footprint 44is centred upon each of the points X in turn. When the camera footprint44 is centred upon each point X, one or more images of the imagingtarget 40 are captured by the camera 20. In this embodiment, the turret28 may be controlled such that the camera footprint 44 is moved in thedirection of travel of the aircraft 2, and/or in a direction that isperpendicular to the direction of travel of the aircraft 2. Suchmovement of the camera footprint 44 is indicated in FIG. 9 by arrows andthe reference numerals 52).

In this embodiment, the imaging target 40 comprises a curved portionalong which multiple images are to be taken. This curved portion isindicated in FIG. 9 by a dotted box and the reference numeral 54. Inthis embodiment, the turning radius of the aircraft 2 is larger than theradius of curvature of the curved portion 54 of the imaging target 40.Thus, the second route 50 includes a loop, which is indicated in FIG. 9by a dotted box and the reference numeral 56. In this embodiment, aroute contains a “loop” if, when viewed from a certain direction, e.g.from above, the route crosses itself at at least one point. In thisembodiment, the loop 56 increases the length of time that the aircraftspends in the vicinity of the curved portion 54, thereby allowing theaircraft 2 to image the curved portion in accordance with the secondimaging schedule.

Preferably, the second route 50 is determined so as to minimise thenumber of loops 56. In this embodiment, the radius 58 of the loop 56 isdependent upon the minimum turn radius of the aircraft 2. In particular,the radius 58 is greater than or equal to the minimum turn radius of theaircraft 2.

In other embodiments, the second route 50 has a different shape to thatdescribed above. Also, in other embodiments, the turret 28 is controlledso as to move the camera footprint 44 in a different way to thatdescribed above, while still providing that each point along the lengthof the imaging target 40 is contained within at least one image capturedby the camera 20 as the aircraft 2 follows the second route 50.

In some embodiments, after following the second route 50 and imaging theentirety of the imaging target 40, the processor 12 calculates a furthersecond route and a further second imaging schedule. The further secondroute and the further second imaging schedule may be such that, were theaircraft 2 to follow the further second route and capture images inaccordance with the further second imaging schedule, each point on theground 8 along the length of the linear imaging target 40 would becontained within at least one of the captured images (i.e. a furtherfeature following process would be performed to image the image target40). An advantage provided by performing more than one feature followingprocess on the imaging target 40 is that errors in determinedgeolocations of detected targets tend to be reduced. In particular, whendetermining a geolocation of a detected target from an image (which isdescribed in more detail later below), the uncertainty associated withthat determined geolocation tends to be largest in the direction oftravel of the aircraft 2 when that image was taken. Using more than oneimage to determine a geolocation of a detected target tends toadvantageously decrease the associated uncertainty. Preferably, for eachimage used to determine a geolocation of a target, the direction thatthe aircraft 2 was travelling when that image was taken is different tothe direction that the aircraft 2 was travelling when each of the otherimages used to determine the geolocation of that target was taken.

What will now be described is a third embodiment of an imaging processperformed at step s12.

In this third embodiment, the imaging target 40 is point on the ground 8or a relatively small area of terrain. In the third embodiment, theimaging of the imaging target 40 comprises conducting “surveillance” ofthe imaging target 40. The terminology “surveillance” is used herein torefer to the reconnaissance of a target (e.g. a detected object) or atarget area (i.e. a relatively small defined area of terrain). In thisembodiment, surveillance is a process comprising taking images of theimaging target 40, such that the entirety of the imaging target 40 iscontained in each of those images. In this embodiment, surveillancefurther comprises processing the captured images to detect targets ofinterest within those images.

In some embodiments, the imaging target 40 upon which a surveillanceprocess is performed may be a target that has been previously detected,for example, by performing the wide area search process (as described inmore details above with reference to FIGS. 6 and 7) or the featurefollowing process (as described above with reference to FIGS. 8 and 9).

FIG. 10 is a process flow chart showing certain steps in the thirdembodiment of the imaging process.

At step s64, the processor 12 acquires the specification for the imagingtarget 40 and the specification for the volume of airspace 38 that arestored in the storage module 14.

At step s66, the processor 12 acquires current performance parametervalues for the aircraft 2 from the aircraft subsystems 16. Examples ofappropriate aircraft performance parameter values include, but are notlimited to, velocities at which the aircraft 2 is capable of travelling,altitudes at which the aircraft 2 is capable of travelling, and aturning radius for the aircraft 2

At step s68, the processor 12 acquires performance parameter values forthe sensor module 10 from the interface module 22. Examples ofappropriate performance parameter values for the sensor module 10include, but are not limited to, the frequency with which the camera 20can capture images, a range of motion of the turret 28 (with respect tothe aircraft fuselage), and a maximum speed at which the turret 28 maymove.

In this embodiment, the aircraft 2 is required to perform surveillanceof the imaging target 40 for a pre-specified amount of time. Theprocessor 12 acquires a specification of this time period (e.g. whichmay have been loaded onto the aircraft 2 prior to take-off).

At step s70, using some or all of the information acquired at steps s64,s66, and s68, the processor 12 determines a route, hereinafter referredto as the third route, within the volume of airspace 38 for the aircraft2. Preferably, the third route is such that, were the aircraft 2 tofollow that route, at each time step within the time period, the sensormodule 10 would be capable of capturing an image containing the entiretyof the imaging target 40.

Further information about the third route is described in more detaillater below with reference to FIG. 11.

At step s72, the processor 12 determines an imaging schedule,hereinafter referred to as the third imaging schedule, for the sensormodule 10. In this embodiment, the third imaging schedule specifies thetime-steps of the time period and the imaging target 40.

Each of the sequence of points along the third route may be specified,for example, by an aircraft position (e.g. as GPS coordinates and analtitude). The imaging target 40 of which the camera 20 is to capture animage at each time step of the time period may be specified by GPScoordinates for that target.

Further information about the third imaging schedule is described inmore detail later below with reference to FIG. 11.

At step s74, the processor 12 sends the third imaging schedule to theinterface module 22 of the sensor module 10.

At step s76, the aircraft 2 is controlled (e.g. by the processor 12) soas to follow the third route.

At step s78, as the aircraft 2 follows the third route, the interfacemodule 22 acquires position and orientation measurements from theposition and orientation module 24. In particular, the interface module22 acquires position measurements from the position sensor 30 andorientation measurements from the orientation sensor 32.

In this embodiment, the position and orientation module 24 is fixed tothe support structure 26. Also, the turret 28 is fixed to the supportstructure 26. Thus, using the acquired position and orientationmeasurements, and using a known positional relationship between theturret 28 and the position and orientation module 24, and using theknown orientation of the turret 28 relative to the support structure 26,the interface module 22 is able to determine a current position andorientation for the camera 20.

At step s80, using the determined current position and orientation ofthe camera 20 and using the second imaging schedule received from theprocessor 12, as the aircraft 2 follows the second route, the interfacemodule 22 controls the turret 28 and the camera 20 so as to captureimages in accordance with the third imaging schedule (i.e., at eachtime-step within the time period, capture one or more images that whollycontain the imaging target 40).

For example, for a time step in the time period, using the currentposition and orientation of the camera 20, and using the specificationof the imaging target 40, the interface module 22 determines a positionan orientation for the camera 20 that would provide that the imagingtarget 40 would be wholly located in the camera's footprint on theground 8. The interface module 22 then controls the turret 28 so thatthe camera 20 has the determined position and orientation. Once thecamera 20 has the desired position and orientation, the interface module22 controls the camera 20 to capture one or more images of the imagingtarget 40.

Further information about the capturing of images performed at step s80is described in more detail later below with reference to FIG. 11.

At step s82, the camera 20 sends the captured images to the interfacemodule 22.

At step s84, the interface module 22 processes the received images so asto convert those images into the predetermined format that is usable bythe processor 12.

After step s84, the method proceeds back to step s14 of FIG. 4, at whichpoint the interface module 22 sends the converted images to theprocessor 12.

Thus, a third embodiment of the imaging process is provided.

FIG. 11 is a schematic illustration (not to scale) of the aircraft 2performing surveillance of the imaging target 40, as described abovewith reference to FIG. 10.

In FIG. 11, the third route is indicated by the reference numeral 58.The direction of travel of the aircraft 2 along the third route 58 isindicated in FIG. 11 by arrow heads placed along the third route 58.

As in FIGS. 7 and 9, in FIG. 11, the camera footprint (i.e. the ground 8that is able to be imaged by the camera 20 at a particular time-step) isindicated by the reference numeral 44.

In this embodiment, the size of the camera footprint 44 on the ground 8at a particular time-step is dependent on the position and orientationof the camera 20 relative to the aircraft 2 (which may be controlled bycontrolling the turret 28), the position and orientation of the aircraft2 (including the altitude of the aircraft 2 above the ground 8), and thesurface relief of the ground 8 (which may, for example, be loaded ontothe aircraft 2 prior to take-off). The size of the camera footprint 44on the ground 8 at a particular time-step may be determined by theprocessor 12.

In this embodiment, the third 58 route is such that, at each point alongthe third route 58, the sensor module 10 on-board the aircraft 2 is ableto capture an image that wholly contains the imaging target 40. In thisembodiment, the third route 58 is an “off-set loiter” whereby, at eachpoint along the third route 58, the distance between the aircraft 2 andthe imaging target 40 is greater than or equal to a predeterminedminimum distance. This predetermined minimum distance may, for example,be uploaded onto the aircraft 2 prior to the aircraft 2 taking off, andstored in the storage module 14. An off-set loiter type routeadvantageously tends to reduce the likelihood of the aircraft 2 beingdetected by the entities at or proximate to the imaging target 40compared to a type of route that permits the aircraft 2 to circle abovethe imaging target 40.

In this embodiment, the third route 58 is a loop, i.e. a start point ofthe third route 58 has the same position as an end point of the thirdroute 58. Thus, the aircraft is able to “loiter” relative to theimagining target by following the loop.

Also, the third route 58 may be determined such that, for each pointalong the third route 58, the distance between the aircraft 2 and theimaging target 40 is less than or equal to a predetermined maximumdistance. This predetermined maximum distance may, for example, beuploaded onto the aircraft 2 prior to the aircraft 2 taking off, andstored in the storage module 14. This predetermined maximum distance maybe dependent upon the capabilities of the camera 20 such that, at eachpoint along the third route 58, the camera 20 i capable of capturingimages of the imaging target 40.

In this embodiment, the aircraft 2 comprises an exhaust from which,during flight, waste gases or air from an aircraft engine are expelled.The exhaust of the aircraft points in certain direction relative to theaircraft fuselage. The direction in which the exhaust of the aircraft 2points is the direction in which waste gases from the engine areexpelled. A specification of this direction may be acquired by theprocessor 12, for example, from an aircraft subsystem 16 (e.g. apropulsion system). In this embodiment, the determination of the thirdroute 58 comprises minimising the duration for which the exhaust of theaircraft 2 is directed towards the imaging target 40. In other words, inthis embodiment, the third route 58 is such that the length of time thatthe exhaust is directed towards the imaging target 40 during the thirdroute 58 is minimised. The exhaust of the aircraft 2 tends to produce ahigh level of noise in the direction in which waste gases from theaircraft engine are expelled (compared to the level of noise in otherdirection). Also, the exhaust of the aircraft 2 tends to produce a highlevel of noise compared to other aircraft systems. In some situations,minimising the duration for which the exhaust is directed towards theimaging target 40 may minimise the level of aircraft noise experiencedby entities at or proximate to the imaging target 40. This tends toreduce the likelihood of the aircraft 2 being detected, as a result ofthe noise generated by the aircraft 2, by the entities at or proximateto the imaging target 40.

In this embodiment, the aircraft subsystems 16 include one or moresensors for measuring a speed and direction of wind relative to theaircraft 2. Such measurements may be acquired by the processor 12. Inthis embodiment, the determination of the third route 58 comprises usingmeasurements of the wind relative to the aircraft 2 so as to providethat, at each point along the third route 58, the aircraft 2 is downwindof the imaging target 40. In other embodiments, wind measurements may beused to determine a route such that, at each point along that route, thewind does not carry sound generated by the aircraft 2 (e.g. by theaircraft engine or exhaust) towards the imaging target 40. This tends toreduce the likelihood of the aircraft 2 being detected, as a result ofthe noise generated by the aircraft 2, by the entities at or proximateto the imaging target 40.

In some embodiments, the processor 12 uses wind measurements todetermine the volume of airspace 38 in which the aircraft 2 is permittedto fly whilst following the third route 58. For example, the volume ofairspace 38 may be determined as a volume that is wholly downwind of theimaging target 40.

In some embodiments, the processor 12 determines a position of the Sunrelative to the aircraft 2. This may be performed using a clockmeasurement, a measurement of the location of the aircraft 2, and ameasurement of the orientation of the aircraft 2, each of which may beacquired by the processor 12. In some embodiments, the determination ofan aircraft route may comprise using the determined position of the Sunrelative to the aircraft 2 to reduce glare in the images taken by thecamera 20 and/or increase the likelihood of high quality images of theimaging target 40 being captured. In some embodiments, the determinationof an aircraft route may comprise using the determined position of theSun relative to the aircraft 2 to reduce the likelihood of the aircraft2 being seen by a particular entity, for example, by positioning theaircraft 2 between the Sun and that entity.

What will now be described is an embodiment of the image processingmethod performed by the processor 12 at step s16.

FIG. 12 is a process flow chart showing certain steps of an embodimentof the image processing method.

At step s86, each image received by the processor 12 from the camera 20is “geolocated”. The terminology “geolocate” is used herein to refer toa process by which the real-world position of an image is determined.

In this embodiment, geolocation of an image comprises determining thereal-world coordinates of each corner of the image, thereby determiningthe location of the portion of the ground 8 contained within that image.The coordinates of a corner of an image are determined by the processor12 using the location and orientation of the aircraft 2 when that imagewas taken, and using the position and orientation of the camera 20 withrespect to the aircraft 2 when that image was taken.

The processor 12 may also estimate, for each image, an uncertaintyassociated with the geolocation information for that image.

The processor 12 may also determine, for each image, a time at whichthat image was captured.

At step s88, each image and respective geolocation information (i.e.real-world coordinates of the image corners) is stored in the storagemodule 14.

At step s90, the processor 12 performs a target detection algorithm onthe images of the imaging target 40 stored within the storage module 14.The algorithm is performed to detect targets of interest (e.g. vehicles,buildings, people, etc.) within the images.

Any appropriate target detection algorithm may be used. For example, analgorithm that detects image features dependent on the contrast of thosefeatures in the image, or an edge detection algorithm may be used.

At step s92, for each image, and for each target detected in that image,the processor 12 determines a geolocation for that target. A geolocationfor a target within an image may be determined using the geolocationinformation relating to that image and stored in the storage module 14.

The processor 12 may also estimate, for each target, an uncertaintyassociated with the geolocation information for that target. Theprocessor 12 may also determine, for each target, a time at which thatthe image of that target was captured.

At step s94, for each image, a list of the targets detected within thatimage and the corresponding geolocation information for the targets iscompiled.

At step s96, the lists of the target (including the geolocationinformation for the detected targets) are stored in the storage module14.

At step s98, image property information (including the geolocationinformation of each of the images and, in some embodiments, informationabout the errors/uncertainty associated with that geolocationinformation and/or times at which each of the images were taken) istransmitted from the aircraft 2 to the ground station 4, by thetransceiver 18, via the wireless communications link 6.

At step s100, target property information (including the geolocationinformation for each of the detected targets and, in some embodiments,information about the errors/uncertainty associated with thatgeolocation information and/or times at which images of each of thetargets were taken) is transmitted from the aircraft 2 to the groundstation 4, by the transceiver 18, via the wireless communications link6.

In this embodiment, only information relating to certain properties ofthe images/targets is transmitted to the base station from the aircraft2, not the images themselves. Thus, the amount of data transmitted tothe ground station 4 at steps s98 and s100 tends to be small relative tothe amount of data that would be transmitted were the images themselvestransmitted.

In this embodiment, as more images of the imaging target 40 are capturedby the camera 20 and processed by the processor 12, targets detected indifferent images are associated together (i.e. assumed to be the same)if the geolocations of those targets are the same or within apre-defined distance of one another. A geolocation of a target may bedetermined using the geolocations of that target in each of thedifferent images in which that target is detected. For example, thegeolocation of a target may be determined as the average (or centre ofmass) of the geolocations of that target determined from each of thedifferent images in which that target was detected. This process ofassociating together targets with the same or sufficiently similargeolocation information advantageously tends to reduce the uncertainlyabout a detected target's true geolocation.

Thus, in this embodiment, geolocation information for detected targetsis continuously updated during the imaging of the imaging target 40.Updated information may be continuously sent from the aircraft 2 to theground station 4.

In some embodiments, as more images of the imaging target 40 arecaptured by the camera 20, those images may be registered together.

At step s102, the information sent to the ground station 4 at steps s98and s100 is displayed to an operator at the ground station 4 (e.g.target locations may be displayed on a map on a display screen). Timeand date information for an image (specifying when an image was taken)may also be displayed to the operator.

At step s104, the operator selects a particular target of interest (e.g.by selecting that target on the display screen). In this embodiment,this generates a request for an image of the selected target to bereturned to the ground station 4.

The operator may request that a certain type of image is returned to theground station 4. For example, the operator may request a cropped image(i.e. a sub-image) containing a certain target, or a compressed versionof an entire camera image containing that target.

At step s106, the ground station 4 sends the generated request to thetransceiver 18 of the aircraft 2. The transceiver 18 relays the requestto the processor 12.

At step s108, the processor 12 processes the received request andretrieves, from the storage module 14, one or more images containing thetarget specified in the request (i.e. the particular target that wasselected by the operator at step s104).

At step s110, the processor 12 processes the retrieved image such thatan image corresponding to the operator's request is produced.

At step s112, the transceiver 18 transmits the produced image to theground station 4 via the wireless communications link 6.

At step s114, the image received at the ground station 4 is displayed tothe operator for analysis.

Thus, the image processing method performed at step s16 of thetrajectory planning algorithm is provided.

In this embodiment, after a target has been detected by performing theimaging process described above with reference to FIGS. 4 to 12, apayload delivery process is performed so as to deliver the payload 19 toa detected target. In other embodiments, a different process (e.g. adifferent payload delivery process) is performed after a target has beendetected.

FIG. 13 is a process flow chart showing certain steps of an embodimentof a payload delivery process.

At step s116, the operator located at the ground station 4 identifies atarget to which the payload 19 is to be delivered. The target to whichthe payload 19 is to be delivered is hereinafter referred to as the“payload target”. For example, at step s114 of the above describedimaging process, the operator analyses the displayed images and selectsa target within the displayed images as being the payload target.

At step s118, the ground station 4 sends a geolocation of the payloadtarget to the transceiver 18 of the aircraft 2. The transceiver 18relays this target specification to the processor 12. In otherembodiments, a target identifier may be sent to the aircraft 2 and theprocessor 12 may determine/acquire a geolocation for that specifiedtarget using information stored in the storage module 14.

At step s120, the processor 12 acquires current aircraft parametervalues for the aircraft 2 from the aircraft subsystems 16. Examples ofappropriate aircraft parameter values include, but are not limited to,velocities at which the aircraft 2 is capable of travelling, altitudesat which the aircraft 2 is capable of travelling, and a turning radiusfor the aircraft 2. In this embodiment, the aircraft subsystems 16include one or more sensors for measuring a speed and direction of windrelative to the aircraft 2. Such measurements are also acquired by theprocessor 12.

At step s122, the processor 12 acquires values of one or more parametersrelating to the payload 19. In this embodiment, the processor 12acquires values for the mass of the payload 19 and a drag coefficientfor the payload 19 in air (or other value indicative of the drag thatwould be experienced by the payload 19 were the payload 19 to bereleased from the aircraft 2). In this embodiment, the processor furtheracquires other properties of the payload 19 such as the type of payload19, whether or not the payload 19 is a dumb payload, a steered payload,a guided payload, or another type of payload, and whether or not thepayload includes a parachute. The processor 12 may also acquire, e.g.from the storage module 14 or from the payload 19, a specification of adistance from the payload target within which the payload 19 is to landon the ground 8.

At step s124, using some or all of the information acquired by theprocessor 12 at steps s118-s122, the processor 12 determines a locationand a velocity, which are hereinafter referred to as the “payloadrelease location” and “payload release velocity” respectively. Thepayload release location may be specified by a geolocation and analtitude. The payload release velocity may be specified by an aircraftheading and an aircraft speed. In this embodiment, the payload releaselocation and payload release velocity are such that, were the aircraft 2to release the payload 19 whilst located at the payload release locationand travelling with the payload release velocity, the payload 19 wouldland on the ground 8 within the pre-specified distance of the payloadtarget.

In some embodiments, for example in embodiments in which the payload isa steered or guided payload, the payload release location is a volume ofairspace in which the payload may be released (and subsequently steeredor guided, e.g. by the processor 12, towards the payload target). Insuch embodiments, the payload release velocity may be a range ofvelocities.

At step s126, using the determined payload release location and usingmeasurements of the aircraft's current position and orientation, theprocessor 12 determines a route from the aircraft's current location tothe payload release location. This determined route will hereinafter bereferred to as the fourth route.

At step s128, using the determined fourth route, the payload releasevelocity and using a measurement of the aircraft's current velocity, theprocessor 12 determines a velocity profile for the aircraft 2 along thealong the fourth route. In this embodiment, the velocity profile is suchthat, were the aircraft 2 to travel along the fourth route with thedetermined velocity profile, the aircraft 2 would arrive at the payloadrelease location travelling at the payload release velocity.

At step s130, the aircraft 2 is controlled (e.g. by the processor 12) soas to follow the fourth route in accordance with the determined velocityprofile.

At step s132, when the aircraft 2 reaches the payload release location,the processor 12 releases the payload 19 from the aircraft 2. At thepayload release location the aircraft is travelling at the payloadrelease velocity.

At step s134, after being release from the aircraft 2, the payload 19travels towards the payload target, and land on the ground 8 within thepre-specified distance of the payload target. Thus, the payload 19 isdelivered to the payload target.

Thus, a payload delivery process is provided.

An advantage provided by the above described system and method is that aroute that is to be followed by the aircraft is determined on-board theaircraft. Also, the aircraft may be controlled so as to follow thedetermined route by systems located on-board the aircraft. Thus, theaircraft tends to be capable of acting autonomously, i.e. withoutreceiving instructions or control signals from the ground station.

A further advantage provided by the above described system and methodsis that task information, including task parameters, can be uploaded tothe aircraft whilst the aircraft is on the ground (i.e. prior to takeoff), or whilst the aircraft is airborne, thereby allowing for theupdating of task parameters after take-off. Furthermore, certain of thetask parameters can advantageously be measured/determined using othersystems on-board the aircraft. For example, an aircrafts GlobalPositioning System (GPS), or the aircraft's avionic or fuel systems etc.can be used to determine parameters such as the location and orientationof the aircraft, the time of day, and/or how much fuel/time is left tocomplete a task.

An advantage provided by the above described sensor module is thatmeasurements taken by the position and orientation module may be used toaccurately determine a position and orientation of the camera. Thistends to be due to the position and orientation module and the turret towhich the camera is mounted having a fixed position and orientation withrespect to one another as a result of being attached to the rigidstructure. The determined position and orientation of the camera tend tobe more accurate than those that may be produced using conventionalsystems, for example, those systems in which position and orientationmeasurements of an aircraft are used to determine a position andorientation of a camera mounted to that aircraft. Accurate position andorientation measurements of the camera tend to facilitate in theaccurate control of the camera. Furthermore, geolocations of the imagesproduced by the camera, and geolocations for targets detected in thoseimages, tend to be more accurate than those produced using conventionalimaging systems.

A further advantage provided by the above described sensor module isthat the sensor module is modular. The interface module, in effect,isolates the processor from the detailed implementation of the cameraand the position and orientation module. The processor sends imagingcommands to the interface module and, in response, receives image datain a predetermined format. The control of the camera and turret isentirely performed by the interface module.

In the above embodiments, the communications link between the processorand the interface module is standardised.

In some embodiments, an operator may replace a sensor module thatincludes one type of imaging sensor with a sensor module that includes adifferent type of imaging sensor. In other words, a sensor module thatincludes one type of imaging sensor may be removed from the aircraft anda sensor module that includes a different type of imaging sensor may beinstalled in its place. As the communications link between the processorand the interface module is standardised across all such sensor modules,updates to other aircraft systems (such as the processor) tend not to berequired. Sometime after being installed on an aircraft, the interfacemodule of a sensor module may send certain sensor module parameters(such as sensor types, range of motion etc.) to the processor.

In the above embodiments, the sensor module includes the turret.However, in other embodiments, the sensor module does not include thatturret. Thus, when replacing a first sensor module with a second sensormodule, the first sensor module may be removed from the turret and thesecond sensor module may be attached to the turret in its place.

Advantageously, using the above described system and methods, theaircraft tends to be capable of performing a wide area search of a givenarea of terrain. The wide area search of the given area of terrain maybe performed autonomously by the aircraft. The wide area search of thegiven area of terrain advantageously tends to facilitate the detectionof targets within that area of terrain. Furthermore, advantageously, thewide area search may be performed such that a number of criteria aresatisfied (e.g. such that the number of turns performed by the aircraftwhile performing the wide area search is minimised).

Advantageously, using the above described system and methods, theaircraft is able to follow (for example, fly above) an elongate portionof terrain, and capture images along the entire length of that elongateportion of terrain. The feature following process may be performedautonomously by the aircraft. The feature following processadvantageously tends to facilitate the detection of targets along thelength of the elongate region of terrain. Furthermore, advantageously,the aircraft may follow the elongate portion of terrain even if theelongate portion of terrain includes bends or curves that have a radiusof curvature that is smaller than the turning radius of the aircraft.This tends to be provided by including one or more loops in theaircraft's route.

Advantageously, using the above described system and methods, theaircraft tends to be capable of performing surveillance of a target onthe ground. The surveillance may be performed autonomously by theaircraft. The surveillance of a target advantageously tends tofacilitate the detection of other targets at or proximate to targetunder surveillance. For example, if the target under surveillance is abuilding, the above described surveillance process may be performed todetect (and subsequently identify) people of vehicles entering orleaving that building. In some embodiments, the surveillance of a targetmay be performed to detect actions performed by that target. Forexample, if the target under surveillance is a vehicle, the abovedescribed surveillance process may be performed to detect when thatvehicle moves, and where that vehicle moves to.

Advantageously, the surveillance process may be performed such that anoise signature of the aircraft experienced at or proximate to thetarget under surveillance tends to be minimised. This advantageouslytends reduce the likelihood of the aircraft being detected by entitieslocated at or proximate to the target under surveillance.

An advantage provided by performing the above described image processingmethod is that, unless otherwise instructed, the aircraft only transmitsimage properties (i.e. image geolocation and the associated uncertaintyetc.), and the properties of any detected targets. In other words,unless such data is requested, complete image data is not transmittedfrom the aircraft to the ground station. The image/target property datatends to be a much smaller amount than complete image data. Thus,bandwidth requirements of communications between the aircraft and theground station tend to be reduced.

Furthermore, only the image data of particular interest to an operatorat the ground station (i.e. only cropped sub-images or compressed imagesthat are requested by the operator) are transmitted to the groundstation for analysis. This further tends to provide that bandwidthrequirements of communications between the aircraft and the groundstation are reduced. Moreover, since relatively useless and/or redundantinformation is not transmitted to the operator for analysis, theanalysis by the operator tends to be easier and/or more efficient.

A further advantage provided by the above described image processingmethod is that image information and information about any detectedtargets (e.g. geolocation etc.) tends to be continuously updated as moreimages are taken by the aircraft. This advantageously tends to reduceuncertainty in the information provided to the ground station. Thus,more accurate results tend to be produced compared to conventional imageprocessing techniques.

Advantageously, the above described payload delivery process may be usedto deliver a payload to a target. The aircraft tends to be capable ofdelivering the payload to its intended target autonomously.Environmental conditions, such as wind speed and direction, and also thepresence of terrain features (such as lakes, rivers, mountains, etc.)may advantageously be taken into account during the payload deliveryprocess. Advantageously, the processor tends to be capable ofdetermining an optimum aircraft position and velocity for payloadrelease.

Apparatus, including the processor and/or the interface module, forimplementing the above arrangement, and performing the above describedmethod steps, may be provided by configuring or adapting any suitableapparatus, for example one or more computers or other processingapparatus or processors, and/or providing additional modules. Theapparatus may comprise a computer, a network of computers, or one ormore processors, for implementing instructions and using data, includinginstructions and data in the form of a computer program or plurality ofcomputer programs stored in or on a machine readable storage medium suchas computer memory, a computer disk, ROM, PROM etc., or any combinationof these or other storage media.

It should be noted that certain of the process steps depicted in any ofthe flowcharts and described herein may be omitted or such process stepsmay be performed in differing order to that presented herein and shownin the Figures. Furthermore, although all the process steps have, forconvenience and ease of understanding, been depicted as discretetemporally-sequential steps, nevertheless some of the process steps mayin fact be performed simultaneously or at least overlapping to someextent temporally.

In the above embodiments, the imaging process is implemented by anunmanned air vehicle. However, in other embodiments a different type ofvehicle is used. For example, in other embodiments, an unmannedland-based vehicle, or a semi-autonomous or manned aircraft is used.

In the above embodiments, a single vehicle images a single imagingtarget. However, in other embodiments a plurality of vehicles is used.Also, in other embodiments, there is a plurality of different imagingtargets.

In the above embodiments, the camera is a visible band camera. However,in other embodiments, a different type of sensor is used. For example,an infrared camera, an ultra-violet camera, a range sensor, or anultrasound sensor may be used. In some embodiments, the sensor moduleincludes more than one type of sensor.

In the above embodiments, the flight path that the aircraft follows fromthe ground station to the volume of airspace is defined by a sequence ofwaypoints. However, in other embodiments the flight path may be definedin a different way, for example, using a sequence of aircraft headingsand corresponding flight durations. In other embodiments, the aircraftmay be controlled by a human operator until the aircraft arrives at apoint in the volume of airspace.

In the above embodiments, the processor determines the route that theaircraft is to follow to perform an imaging process in response to theaircraft entering the volume of airspace. However, in other embodiments,the processor determines the route when a different set of criteria havebeen satisfied. For example, in other embodiments the route for theimaging process is determined by the processor when the aircraft is at aspecific location, within a pre-determined distance of a specificlocation, or at a certain time of day.

In the above embodiments, a volume of airspace is defined in which theaircraft is permitted to fly whilst performing the imaging process.However, in other embodiments no such volume is defined. For example, inother embodiments the aircraft is allowed to fly anywhere during theimaging process. In some embodiments, a minimum distance that theaircraft must be from the imaging target while performing the imagingprocess is implemented. In some embodiments, a maximum distance that theaircraft may be from the imaging target while performing the imagingprocess is implemented.

A route that the aircraft is to follow to perform an imaging process maybe any shape. Furthermore, a route may depend on any appropriatecriteria or measurements instead of or in addition to those mentionedabove. For example, a requirement that the aircraft remainssubstantially at certain compass bearing from the area of terrain may beimplemented.

In the above embodiments, the aircraft performs a single imagingprocess. However, in other embodiments a different number of imagingprocesses are performed. One or more of the performed imaging processesmay be different to one or more of the other imaging processes that areperformed. For example, in some embodiments, one or more wide areasearches and/or one or more feature following processes may be performedto detect a target within a certain region. One or more surveillanceoperations may then be performed on the detected target.

In the above embodiments, during the information processing process,data is transmitted to the ground station from the aircraft for analysisby an operator. However, in other embodiments, data is transmitted fromthe aircraft to a different entity, for example, an entity that isremote from the aircraft such as a different aircraft. In someembodiments, data is transmitted from the processor for use by othersystems on-board the aircraft. In some embodiments transmitted data isfor another purpose instead of or in addition to analysis by an operator(e.g. for use as an input to a further process).

In the above embodiments, a payload delivery process is performed todeliver a single payload to a single target. However, in otherembodiments, the payload delivery process may be performed to deliver adifferent number of payloads to a different number of targets. In someembodiments, there may be a plurality of different types of payloads.

In the above embodiments, captured images are processed using the imageprocessing method described above with reference to FIG. 12. However, inother embodiments images may be processed in a different way. Forexample, in other embodiments an image processing method in which allfull image data (i.e. all data gathered by the aircraft) is transmittedto the ground station is used. Also, in other embodiments, an imageprocessing method in which no data is transmitted whilst the aircraft isairborne is used. For example, all image data may be stored on-board theaircraft and be downloaded when the aircraft lands.

The invention claimed is:
 1. An imaging method for imaging an area ofterrain using a sensor mounted on an unmanned aircraft, the methodcomprising: acquiring, by one or more processors, a specification ofpossible positions and orientations relative to the aircraft to whichthe sensor may be moved; acquiring, by the one or more processors,positional information of the area of terrain; acquiring, by the one ormore processors, a specification of the manoeuvrability of the aircraft;using the acquired specification of the possible positions andorientations of the sensor relative to the aircraft, the acquiredpositional information of the area of terrain, and the acquiredspecification of the manoeuvrability of the aircraft, determining, bythe one or more processors, a first procedure and a second procedure;performing, by the aircraft via one or more aircraft subsystems and theone or more processors, the first procedure, the first procedurecomprising the aircraft moving with respect to the area of terrain alonga first route and the sensor moving with respect to the aircraft suchthat, for each point in the area of terrain, each point is coincidentwith a footprint of the sensor on the ground for at least some timeduring the first procedure; whilst the aircraft performs the firstprocedure, capturing, by the sensor, a first set of images, each imagein the first set being of only part of the area of terrain, the firstset of images being such that, for every point in the area of terrain,one of the points is present within at least one of the images in thefirst set; processing the first set of images to detect, within at leastone of the first images, a first target; and acquiring, by the one ormore processors, a position on the ground of the detected first target;thereafter, performing, by the aircraft via said one or more aircraftsubsystems and the one or more processors, the second procedure, thesecond procedure comprising the aircraft moving with respect to the areaof terrain along a second route and the sensor moving with respect tothe aircraft such that, for each point in the area of terrain, eachpoint is coincident with a footprint of the sensor on the ground for atleast some time during the second procedure, wherein the secondprocedure is determined using the position on the ground of the firsttarget, such that a direction in which the aircraft flies when the firsttarget is imaged during the second procedure is perpendicular to adirection in which the aircraft was flying during the first procedurewhen an image of the first target was captured; and whilst the aircraftperforms the second procedure, capturing, by the sensor, a second set ofimages, each image in the second set being of only part of the area ofterrain, the second set of images being such that, for every point inthe area of terrain, one of the points is present within at least one ofthe images in the second set.
 2. A method according to claim 1, furthercomprising registering the first set of images with the second set ofimages.
 3. A method according to claim 1, wherein determining the secondprocedure comprises minimising overlap between the first route and thesecond route.
 4. A method according to claim 1, wherein the one or moreprocessors are located on-board the aircraft.
 5. A method according toclaim 1, wherein the method further comprises acquiring, by the one ormore processors, a specification of a volume of airspace; the step ofdetermining the first procedure comprises, using the specification ofthe volume of airspace, determining the first procedure such that theaircraft remains with the volume of airspace during the first procedure.6. A method according to claim 1, wherein the first procedure isdetermined such that the number of turns performed by the aircraftwhilst performing the first procedure is minimised.
 7. A methodaccording to claim 1, the method further comprising: processing thecaptured images to detect, within at least one image, a second target;acquiring, by the one or more processors, a position on the ground ofthe detected second target; using the acquired specification of possiblepositions and orientations relative to the aircraft to which the sensormay be moved, the acquired position of the second target, and thespecification of the manoeuvrability of the aircraft, determining, bythe one or more processors, a third procedure to be performed by theaircraft; performing, by the aircraft via said one or more aircraftsubsystems and the one or more processors, the third procedure; andwhilst the aircraft performs the third procedure, capturing, by thesensor, a third set of images; wherein the third procedure comprises theaircraft moving with respect to the second target and the sensor movingwith respect to the aircraft such that the second target is coincidentwith a footprint of the sensor on the ground for the entire duration ofthe third procedure; and capturing the third set of images is performedsuch that the whole of the second target is present within each image inthe third set.
 8. A method according to claim 1, the method furthercomprising: processing the captured images to detect, within at leastone image, a third target; acquiring, by the one or more processors, aposition on the ground of the detected third target; acquiring, by theone or more processors, a specification of a direction relative to theaircraft in which an exhaust of the aircraft points; and using theacquired position of the third target, the specification of themanoeuvrability of the aircraft, and the acquired specification of thedirection, determining by the one or more processors, a third route forthe aircraft; and following, by the aircraft, the third route; whereinthe determination of the third route comprises minimising a duration forwhich the exhaust of the aircraft is directed towards the third target.9. A method according to claim 1, wherein the aircraft comprises apayload releasably attached to an aircraft; and the method furthercomprises: processing the captured images to detect, within at least oneimage, a fourth target; acquiring, by the one or more processors, aposition on the ground of the detected fourth target; acquiring, by theone or more processors, parameter values relating to properties of thepayload; acquiring, by the one or more processors, parameter valuesrelating to environmental conditions in which the aircraft is flying;using the acquired position of the fourth target, the acquired parametervalues relating to properties of the payload, and the acquired parametervalues relating to environmental conditions, determining, by the one ormore processors, a position and a velocity for the aircraft; using thedetermined position and velocity for the aircraft, determining, by theone or more processors, a fourth procedure for the aircraft; performing,by the aircraft via said one or more aircraft subsystems and the one ormore processors, the fourth procedure; and at a point in the fourthprocedure that the aircraft has the determined position and velocity,releasing, by the aircraft, the payload; the determined position and avelocity for the aircraft are such that, were the aircraft to releasethe payload whilst located at the determined position and travelling atthe determined velocity, the payload would land on the ground within apredetermined distance of the fourth target; and the fourth procedure issuch that, were the aircraft to perform the fourth procedure, at atleast one instance during the fourth procedure, the aircraft would belocated at the determined position and travelling at the determinedvelocity.
 10. A method according to claim 1, the step of capturing theset of images comprises, for each image: acquiring, by one or moreprocessors, a specification of a region on the ground to be imaged;measuring, by a position sensor fixedly mounted to a rigid supportstructure, a position of the position sensor; measuring, by anorientation sensor fixedly mounted to the rigid support structure, anorientation of the orientation sensor; using the measured position andorientation and using the acquired region specification, determining aposition and orientation for the sensor, the sensor being mounted to therigid support structure; controlling the aircraft and the orientation ofthe sensor on-board the aircraft such that the sensor has the determinedposition and orientation, thereby providing that a footprint of thesensor on the ground is coincident with the region on the ground to beimaged; and when the sensor has the determined position and orientation,capturing, by the sensor, one or more images of the area of the groundwithin the sensor footprint; wherein the rigid support structure isreleasably coupled to the aircraft of the aircraft.
 11. Apparatus forimaging an area of terrain the apparatus comprising: a sensor mountedon-board an aircraft; one or more processors configured to: acquire aspecification of possible positions and orientations relative to theaircraft to which the sensor may be moved; acquire positionalinformation of the area of terrain; acquire a specification of themanoeuvrability of the aircraft; using the acquired specification ofpossible positions and orientations relative to the aircraft to whichthe sensor may be moved, the acquired positional information of the areaof terrain and the acquired specification of the manoeuvrability of theaircraft, determine a first procedure and a second procedure; and one ormore aircraft subsystems and the one or more processors for controllingthe aircraft to perform the first procedure and, thereafter, the secondprocedure; wherein the first procedure comprises the aircraft movingwith respect to the area of terrain along a first route and the sensormoving with respect to the aircraft such that, for each point in thearea of terrain, each point is coincident with a footprint of the sensoron the ground for at least some time during the first procedure; thesecond procedure comprises the aircraft moving with respect to the areaof terrain along a second route and the sensor moving with respect tothe aircraft such that, for each point in the area of terrain that pointis coincident with a footprint of the sensor on the ground for at leastsome time during the second procedure; and the sensor is configured to:whilst the aircraft performs the first procedure, capture a first set ofimages, each image in the first set being of only part of the area ofterrain, the first set of images being such that, for every point in thearea of terrain, one of the points is present within at least one of theimages in the first set; process the first set of images to detect,within at least one of the first images, a first target; acquire aposition on the ground of the detected first target, determine thesecond procedure by using the position on the ground of the firsttarget, such that a direction in which the aircraft flies when the firsttarget is imaged during the second procedure is perpendicular to adirection in which the aircraft was flying during the first procedurewhen an image of the first target was captured; and whilst the aircraftperforms the second procedure, capture a second set of images, eachimage in the first set being of only part of the area of terrain, thesecond set of images being such that, for every point in the area ofterrain, that point is present within at least one of the images in thesecond set.
 12. A non-transitory program or plurality of non-transitoryprograms such that when executed by a computer system or one or moreprocessors mounted on an unmanned aircraft it/they cause the computersystem or the one or more processors to: acquire, by one or moreprocessors, a specification of possible positions and orientationsrelative to the aircraft to which the sensor may be moved; acquire, bythe one or more processors, positional information of the area ofterrain; acquiring, by the one or more processors, a specification ofthe manoeuvrability of the aircraft use the acquired specification ofthe possible positions and orientations of the sensor relative to theaircraft, the acquired positional information of the area of terrain,and the acquired specification of the manoeuvrability of the aircraft,determining, by the one or more processors, a first procedure and asecond procedure; perform, by the aircraft via one or more aircraftsubsystems and the one or more processors, the first procedure, thefirst procedure comprising the aircraft moving with respect to the areaof terrain along a first route and the sensor moving with respect to theaircraft such that, for each point in the area of terrain, each point iscoincident with a footprint of the sensor on the ground for at leastsome time during the first procedure; whilst the aircraft performs thefirst procedure, capture, by the sensor, a first set of images, eachimage in the first set being of only part of the area of terrain, thefirst set of images being such that, for every point in the area ofterrain, one of the points is present within at least one of the imagesin the first set; process the first set of images to detect, within atleast one of the first images, a first target; and acquire, by the oneor more processors, a position on the ground of the detected firsttarget; thereafter, perform, by the aircraft via said one or moreaircraft subsystems and the one or more processors, the secondprocedure, the second procedure comprising the aircraft moving withrespect to the area of terrain along a second route and the sensormoving with respect to the aircraft such that, for each point in thearea of terrain, each point is coincident with a footprint of the sensoron the ground for at least some time during the second procedure,wherein the second procedure is determined using the position on theground of the first target, such that a direction in which the aircraftflies when the first target is imaged during the second procedure isperpendicular to a direction in which the aircraft was flying during thefirst procedure when an image of the first target was captured; andwhilst the aircraft performs the second procedure, capture, by thesensor, a second set of images, each image in the second set being ofonly part of the area of terrain, the second set of images being suchthat, for every point in the area of terrain, one of the points ispresent within at least one of the images in the second set.
 13. Thenon-transitory program or plurality of non-transitory programs accordingto claim 12 wherein said non-transitory program or plurality ofnon-transitory programs are disposed on a non-transitory machinereadable storage medium.